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Introduction: The Department of Health and Human Services and Food and Drug Administration 
described guidelines for exception from informed consent (EFIC) research. These guidelines require 
community consultation (CC) events, which allow members of the community to understand the 
study, provide feedback and give advice. A real-time gauge of audience understanding would allow 
the speaker to modify the discussion. The objective of the study is to describe the use of audience 
response survey (ARS) technology in EFIC CCs. 

Methods: As part of the Rapid Anticonvulsant Medication Prior to Arrival Trial (RAMPART), 13 
CC events were conducted. We prepared a PowerPoint™ presentation with 4 embedded ARS 
questions, according to specific IRB guidelines to ensure that the pertinent information would 
reach our targeted audience. During 6 CCs, an ARS was used to gauge audience comprehension. 
Participants completed paper surveys regarding their opinion of the study following each CC. 

Results: The ARS was used with minimal explanation and only one ARS was lost. Greater than 
80% of the participants correctly answered 3 of the 4 ARS questions with 61 % correctly answering 
the question regarding EFIC. A total of 105 participants answered the paper survey; 80-90% of the 
responses to the paper survey were either strongly agree or agree. The average scores on the paper 
survey in the ARS sites compared to the non-ARS sites were significantly more positive. 

Conclusion: The use of an audience response system during the community consultation aspects 
of EFIC is feasible and provides a real-time assessment of audience comprehension of the study 
and EFIC process. It may improve the community's opinion and support of the study. [West J Emerg 
Med. 2014;15(4):414-418.] 



INTRODUCTION 

There is a critical need for research in certain emergency 
medical conditions to improve outcomes. Given the acute 
nature of some of these conditions, such as sudden cardiac 
arrest, strokes, and status epilepticus, obtaining an informed 
consent may be impossible. 12 To address this ethical issue, the 



Department of Health and Human Services (DHHS) and the 
Food and Drug Administration (FDA) described guidelines 
in 1 996 to allow research to be carried out with an exception 
from informed consent (EFIC). 3 The federal policies require 
that the research subject be in a life -threatening condition for 
which available modalities of treatments are thought to be 
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unsatisfactory. The potential subject must be unable to consent 
and there is no time to contact the legal representative. For 
the subject to take part in the study, there must be a possibility 
that the subject will benefit. 4 " 5 The federal regulation that 
governs EFIC - 21 CFR 50.24 - requires that both community 
consultation (CC) and public disclosure (PD) occur in that 
local community and that local institutional review board 
(IRB) approval be obtained prior to EFIC activities and 
enrollment of subjects. 6 Public disclosure requires that 
researchers inform the community that a study will be taking 
place, usually done through mass media, such as the internet, 
television and radio advertisements. Community consultation 
is designed to allow members of the community and stake 
holders to understand the study, provide feedback and act in 
an advisory role. 7 It is also an opportunity to hear concerns, 
suggestions and questions that may have not been considered. 
An important part of community consultation is to sufficiently 
educate the community members about the study so that 
they are able to provide meaningful feedback. 2 A lack of 
understanding may complicate the evaluation of community 
feedback by the investigators and IRBs. It can be difficult 
for the speaker to gauge the level of audience understanding. 
A real- time gauge of audience understanding would allow 
the speaker to modify the presentation, reiterate important 
aspects of the study, and ensure appropriate communication. 
Several studies have been published on the role of community 
consultation and how the general population views them, 
but we have not found evidence that demonstrates that the 
community groups understand the proposed study. 



1- Which of the following are risks of the study? 
A- Allergic reaction 

B- Slowed breathing 
C-Pain at the injection site 
DAN of the above 

2- We will start the study without the consent of the patient. 
A- True 

B- False 

3- Which of the following people will NOT be enrolled? 
A- Prisoners 

B- known pregnant women 

C- Any person who opts out with a bracelet 

D- All of the above 

4- Can I opt out of the study? 
A- Yes 

B- No 

Figure 1. Audience response questions. 



Question 

This study offers more benefit than harm for the seizing patient 

I would be willing to participate in such a research study if I were 
having a serious seizure and unable to give my permission 

If a family member was enrolled in this study and I was 
told about it afterwards, I would agree to their continued 
participation, until they could consent for themselves 

Are you supportive of this study being done in your community? 



Figure 2. Paper survey questions regarding community members' 
opinion of study. 

An audience response system (ARS) is an electronic 
device that creates an interaction between the presenter and his 
audience in real time and allows for immediate feedback. ARS 
have been playing an important role in medical education and 
have been shown to have more student appeal and satisfaction 
than didactic sessions in continuing medical education 
learners. 8 9 This paper describes the novel use of audience 
response systems in community consultation for EFIC. 

METHODS 
Study Design 

This study describes the use of an ARS during EFIC 
community consultations for a single site of the RAMPART 
study. 10 We retrospectively compared ARS and non-ARS 
CC sites for potential differences in baseline demographic 
characteristics and average rank scores of the final paper 
survey. Our IRB approved this study as a part of the 
RAMPART EFIC process. 

Study Setting and Population 

During the EFIC process, our site carried out 1 3 CC 
events in the spring of 2009 and winter of 20 1 0 in the 
local metropolitan area. CCs were organized with various 
community groups in the greater metropolitan area, such as 
churches, schools, and support groups. Six of the community 
consultations used the ARS presentations. The ARS and non- 
ARS consultations were chosen based on convenience. 

Study Protocol 

We prepared a PowerPoint™ presentation with audience 
response technology from Turning Point Technologies, Inc 
(Youngstown, OH) according to specific IRB guidelines to 
ensure that the pertinent information would reach the targeted 
audience. We embedded the slide set with 4 ARS questions 
that were thought to address the most important issues a 
community needs to understand prior to the study's start 
(Figure 1). At the ARS sites, we used the pooled answers to 
the embedded questions to guide any necessary additional 
explanations of the study during the presentation. The same 
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Table 1 . Demographics of community members attending a 
community consultation event. 

ARS sites Non ARS sites p-value 



Participants 

Mean age 
Female 



76 

49.9 ± 14.2 
38 (49%) 



29 

41.8 ± 13.8 
22 (76%) 



0.011 
0.014 



Table 2. Audience response data summary. 



Question 



% Correct (Total N) 



Which of the following are risks of the 81 % (80) 

study? 

We will start the study without the 61 % (80) 

consent of the patient. 

Which of the following people will 92% (86) 

NOT be enrolled? 

Can I opt out of the study? 93% (87) 



White 


21 (28%) 


6 (21%) 


Hispanic or Latino 


0 (0%) 


1 (3%) 


African American 


46 (61%) 


22 (76%) 


Asian 


5 (7%) 


0 (0%) 


Pacific Islander 


1 (1%) 


0 (0%) 


Native American 


0 (0%) 


0 (0%) 


Other 


3 (4%) 


0 (0%) 


No school 


0 (0%) 


0 (0%) 


Elementary 


0 (0%) 


(0%) 


Some high school 


2 (3%) 


4 (14%) 


High school 


7 (12%) 


3 (10%) 


Some college 


21 (35%) 


8 (28%) 


College 


30 (50%) 


14 (48%) 


ARS, audience response system 



slide presentation was given to all of the ARS consultations. 

At the end of all of the CCs, an anonymous final written 
survey was completed by individuals to obtain feedback 
from the audience and to gauge whether the audience was 
supportive of the study and willing to participate in it. The 
survey collected basic demographic information, qualitative 
comments, and had 4 questions related to opinions about the 
study with a Likert scale (Figure 2). Not all of the CCs were 
amenable to a slide presentation and the ARS slide set was 
not used there. Also, every participant did not complete a final 
paper survey. 

Data Analysis 

We reported age as mean and standard deviation in both 
groups and compared it using a 2-sided 2-sample t-test. 
Gender and ethnicity were reported as frequency and percent 
for each category and compared between groups using a 
Chi-square test and a Fisher's exact test, respectively. We 
evaluated educational level using a 2-sided Cochran-Armitage 
trend test to determine whether or not there was a significant 
trend in education level when compared between the 2 groups. 
The 4 questions related to opinions about the study were 
assigned ranked values from 1 to 4 (1: strongly disagree, 2: 
disagree, 3: agree, and 4: strongly agree). The average rank 
scores for ARS and non ARS were reported for each question 



and compared between groups using 2-sided Wilcoxon rank- 
sum tests. 

RESULTS 

The investigators presented information on the 
RAMPART study to 13 groups during the CC process. At 6 
of these meetings, the presentation was augmented by the 
use of ARS to understand audience comprehension. The ARS 
process required minimal explanation, and only 1 unit was 
lost. There was a significantly higher average age and lower 
proportion of females in the ARS group compared to the non- 
ARS group. There was no significant difference in ethnicity or 
educational level between the 2 groups (Table 1). 

During the ARS presentations, greater than 80% of the 
participants correctly answered 3 of the 4 questions. However, 
the question referencing EFIC and enrollment without the 
patient's infonned consent was correctly answered 61% of the 
time. The ARS results for these 6 groups are shown in Table 2. 

The final paper surveys were intended to obtain written 
feedback from the community regarding the study. Overall, the 
survey respondents had an 80-90% favorable response (strongly 
agree or agree) to the questions presented. Agreement scores 
were significantly higher in the ARS sites compared to the non- 
ARS sites for 3 of the 4 questions (Table 3). 

DISCUSSION 

Since the ARS technology was invented in the 1960s, it 
has been used in evaluating the response of large audiences. As 
the technology advanced, its application increased to cover a 
broad range of industries and organizations, such as marketing, 
corporate training, game shows, universities and continuing 
medical education. 9 Miller et al evaluated the role of ARS for 
the continuing education of health professionals and found that 
ARS enhanced audience attention and learning. 8 Homme et 
al concluded that ARS significantly increased attendance and 
participation during weekly residency board review courses, and 
the residents perceived that the experience was educational." 
During the RAMPART CCs, ARS was found to be a feasible 
method to measure the audience's understanding of the 
presentation. From a financial aspect, it may be a cost-effective 
investment to ensure the message is understood. Furthennore, 
ARS is an easy way to automatically capture data, allowing 
the researcher to bypass the tedious data entry process that is 
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Table 3. Paper survey question with results for sites that used audience response versus those that did not. 

Question Strongly agree Agree (3) Disagree (2) Strongly Average p-value 



(4) disagree (1) score 



This study offers more 
benefit than harm for the 
seizing patient 


ARS sites 
76(100%) 
Non ARS sites 
28 (96%) 


36 (47%) 
7 (26%) 


36 (47%) 
18 (67%) 


3(4%) 
2 (7%) 


1(1%) 
1 (4%) 


3.4 
3.1 


0.035 


I would be willing to 
participate in such a 
research study if I were 
having a serious seizure 
and unable to give my 
permission 


ARS sites 
75 (98%) 
Non ARS sites 
29 (100%) 


30 (38%) 
10(34%) 


34 (47%) 
14 (48%) 


9(12%) 
3(10%) 


2 (3%) 
2 (7%) 


3.2 
3.1 


0.661 


If a family member was 
enrolled in this study 
and I was told about 
it afterwards, I would 
agree to their continued 
participation, until 
they could consent for 
themselves 


ARS sites 

76(100%) 
Non ARS sites 
29(100%) 


32 (42%) 
5(17%) 


42 (55%) 
21 (72%) 


1 (1%) 
1 (3%) 


1 (1%) 

2 (7%) 


3.4 
3.0 


0.010 


Are you supportive of 
this study being done in 
your community? 


ARS sites 
79 (100%) 
Non ARS sites 

29(100%) 


43 (57%) 
8 (30%) 


30 (39%) 
20 (67%) 


2 (3%) 
0 (0%) 


1 (1%) 
1 (3%) 


3.5 
3.2 


0.024 



ARS, audience response system 



necessary when written surveys are used. ARS also works for 
various size audiences. It is convenient for large audiences and 
also allows for anonymous feedback in small groups. 12 

The ultimate goal of community consultation is to provide 
the audience with an advisory role through which they can 
channel concerns, questions and feedback to the IRB and the 
study investigators. 13 Based on their feedback, the IRB and 
investigators can modify some aspects of the protocol to protect 
the potential subjects. In a 2007 review article on CCs, Baren 
et al asked if the community grasped this advisory concept. 14 
While most of the literature addresses how CCs should be 
organized, and how much information should be divulged 
during the event, no literature has addressed the issue of 
audience comprehension. 15 The local community enrolled in 
this study was primarily composed of minorities and African 
Americans. This subset of the population has been shown 
to be reluctant to participate in clinical trials due to previous 
experience with research, such as the Tuskegee experiments. 16 
Through the use of ARS, the speakers were able to identify that 
40% of the audience at the CCs did not understand the concept 
of EFIC and that the study would be started without infonned 
consent. Therefore, the presenters were able to review and 
reiterate this information. This ultimately may have led to a 
greater understanding of the presentation's content and overall 



purpose of the study and more support from the community. 
Alternatively, the ARS group may have felt more engaged and 
therefore more likely to support the study For a different study, 
better understanding may possibly lead to less support if there 
are concerns with the study protocol. 

To make the CCs more credible to the audience, the 
presentations were given by a physician. While a physician 
would have the knowledge to answer most of the audience's 
questions, he/she can also be a source of intimidation to a lay 
person. While questions were encouraged during CCs, a lay 
person may be reluctant to ask simple questions to a physician. 
Similarly, given the large size of audience at CCs, an individual 
person might be reluctant to voice an opinion in such a large 
forum. 14 However, through the use of the ARS, the audience has 
an anonymous method to voice their opinion. 12, 13 

LIMITATIONS 

There are many limitations to this study We did not 
prospectively design the CCs to compare ARS and non- 
ARS sites. Therefore, the 2 groups are not matched cohorts 
and we cannot draw definitive conclusions regarding their 
comparison. Also, we did not assess understanding of the 
study during the final paper survey; therefore we cannot 
comment on the effectiveness of either presentation with 
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regards to comprehension. 

Also, not all of the participants completed a paper survey, 
and everyone at the ARS sites may not have answered the 
ARS questions. Therefore, we do not have data to determine 
the overall response rate for the ARS presentations or the final 
survey. 

Due to the anonymous nature of the paper surveys and the 
ARS responses we could not directly compare the responses 
of individuals. 

CONCLUSION 

The use of an audience response system during the 
CC aspects of EFIC is feasible, and provides a real-time 
assessment of audience comprehension of the study and EFIC 
process. It may improve the community's opinion and support 
of the study. This article will hopefully inspire future research 
in improving community consultations. 
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